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Abstract 

We study a supersymmetric SO(10) gauge theory in six dimensions compactified 
on an orbifold. Three sequential quark-lepton families are localized at the three 
fixpoints where SO(10) is broken to its three GUT subgroups. Split bulk multiplets 
yield the Higgs doublets of the standard model and as additional states lepton 
doublets and down-quark singlets. The physical quarks and leptons are mixtures of 
brane and bulk states. The model naturally explains small quark mixings together 
with large lepton mixings in the charged current. A small hierarchy of neutrino 
masses is obtained due to the different down-quark and up-quark mass hierarchies. 
None of the usual GUT relations between fermion masses holds exactly. 



The explanation of the masses and mixings of quarks and leptons remains a challenge 
for theories which go beyond the standard model [1, 2]. In principle, grand unified theories 
(GUTs) appear as the natural framework to address this question. However, as much work 
on this topic has demonstrated, all simple GUT relations for fermion mass matrices are 
badly violated and, within the conventional framework of four-dimensional (4d) unified 
theories, a complicated Higgs sector is needed to achieve consistency with experiment. 

In this paper we shall address the flavour problem in the context of a supersymmetric 
SO(10) GUT in six dimensions compactified on an orbifold [3, 4]. A new ingredient of 
orbifold GUTs is the presence of split bulk multiplets whose mixings with complete GUT 
multiplets can significantly modify ordinary GUT mass relations [5, 6]. This extends the 
well know mechanism of mixing with vectorlike multiplets [7]. Several analyses of the 
flavour structure of orbifold GUTs have already been carried out (cf., e.g., [8]-[12]). In 
5d theories large bulk mass terms can lead to a localization of zero modes at one of the 
two boundary branes, which can explain fermion mass hierarchies [13]. In this way a 
realistic 'lopsided' structure of Yukawa matrices can be achieved [14]. 

'Lopsided' fermion mass matrices, mostly based on an abelian generation symme- 
try [15], have received much attention in recent years (cf. [16]- [21]). In the context of 
SU(5) GUTs they introduce a large mixing of left-handed leptons and right-handed down 
quarks, which leads to small mixings among the left-handed down-quarks. In this way 
the observed large mixings in the leptonic charged current can be reconciled with the 
small CKM mixings in the quark current. The mechanism of flavour mixing, which we 
describe below, is also based on large mixings of left-handed leptons and right-handed 
down quarks. However, these mixings do not respect SU(5) and they are not controlled 
by a single hierarchy parameter. In this way a different pattern of mixings is achieved 
with several characteristic predictions for the neutrino sector. 

Let us now consider SO(10) gauge theory in 6d with N — 1 supersymme- 
try compatified on the orbifold T 2 /( [3, 4]. The theory has 

four fixed points, Oj, OqG) Ofi and Ops, located at the four corners of a 'pil- 
low' corresponding to the two compact dimensions (cf. fig. 1). At Oj only su- 
persymmetry is broken whereas SO (10) remains unbroken. At Ogg, Ofi and Ops 
SO(10) is broken to its three GUT subgroups G GG =SU(5) xU(l) x , flipped SU(5), 
G/j=SU(5)'xU(l)\ and G P5 =SU(4) xSU(2) xSU(2), respectively. The intersection of all 
these GUT groups yields the standard model group with an additional U(l) factor, 
Gsm'= SU(3)xSU(2)xU(l)yxU(l)x, as unbroken gauge symmetry below the compact- 
ification scale. B — L, the difference of baryon and lepton number, is a linear combination 
of Y and X. 

The field content of the theory is strongly constrained by the required cancella- 
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Figure 1: The three SO(10) subgroups at the corresponding fixpoints of the orbifold 
T*/(Zl x Z 2 PS x Z?C) . 

tion of irreducible bulk and brane anomalies [22]. Motivated by the embedding of all 
field quantum numbers into the adjoint representation of E 8 [23], we have 6 10-plets, 
Hi, . . . , H 6 , and 4 16-plets, $, $ c , 0, C as bulk hypermultiplets, accompanied by 3 16- 
plets fy, i — 1 . . . 3, of brane fields. Vacuum expectation values of $ and <3> c break B — L. 
The electroweak gauge group is broken by expectation values of Hi and H 2 . 

Compared to [23] we have added an additional pair of bulk 16-plets, (ft and (fi c together 
with two 10-plets, H 5 and H$, to cancel bulk anomalies. This is still compatible with 
the embedding in E 8 , and it corresponds to the largest number of bulk fields consistent 
with the cancellation of anomalies. Note that both the irreducible and reducible 6d gauge 
anomalies vanish. 

The parities of if 5, Hq and cf) are listed in table 1. <p c has the same parities as <fi. The 
corresponding zero modes are 

L = ( ^ J , L c = ( "jj , Gl = d% , G 6 = d, . (1) 

The zero modes of the fields $, $ c , Hi . . . H4 are given in [23]. They are the color triplets 
and singlets D c , N c , D, N, H{, H 2 , G c 3 and G A . 

Fermion masses and mixings are determined by brane superpotentials. The allowed 
terms are restricted by R-invariance and an additional U(l)^ symmetry [23]. The cor- 
responding charges of the superfields are given in table 2. The fields H\, H 2 , $ and $ c , 
which aquire a vacuum expectation value, have vanishing R-charge. All matter fields have 
R-charge one. Since tpi and <fi have the same charges we combine them to the quartet 
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Table 1: Parity assignments for the bulk hypermultiplets H 5 , H 6 and (j). 



ijpa) — (^,0), a = 1. . .4. The most general brane superpotential up to quartic terms 
is then given by 



W = M d H 5 H 6 + M l a ij a( p c + M 1 2H 1 H 3 + M23H2H : 



1 h N 

+f D &&H 3 + f G <S><S>H 4 + -^ Q ^$ c $ c 

+^$<$> C H 2 H 3 + ^e-^^i/jff! + ^<$> c ib a H 5 H 2 + —<$>(h c H b H l 
M, M* M* M* 51 

W + + + 5-4 W , (2) 

where we choose M* > ~ ^-gut to be the cutoff of the 6d theory, and the bulk fields 

have been properly normalized. All the volume factors due to the 6d fields are absorbed 
into the unknown couplings and we will not use them to explain the hierarchies. When 
the bulk fields are replaced by their zero modes only 9 of the 23 terms appearing in the 
superpotential remain. Although we have written the superpotential in terms of SO(10) 
multiplets, on the different branes the Yukawa couplings and split into , 
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Table 2: Charge assignments for the symmetries U(1)jj and U(l)^. 



and h( u \h( D \ respectively. Some of these couplings are equal due to GUT relations on 
the corresponding brane. 

The main idea to generate fermion mass matrices is now as follows. We consider the 
case that the three sequential 16-plets are located on the three branes where SO(10) 
is broken to its three GUT subgroups. As an example, we place ipi at Ogg, 1P2 at Ofi 
and vps at Ops- The three 'families' are then separated by distances large compared to 
the cutoff scale M*. Hence, they can only have diagonal Yukawa couplings with the bulk 
Higgs fields. Direct mixings are exponentially suppressed. However, the brane fields can 
mix with the bulk zero modes for which we expect no suppression. These mixings take 
place only among left-handed leptons and right-handed down-quarks. This leads to a 
characteristic pattern of mass matrices which we shall now explore. 

If B — L is broken, as discussed in [23], (<I >C ) = (<&) = v^, and the bulk zero modes 
iV c , N, (D, G c ) and (D c , G) aquire masses 0(vn). After electroweak symmetry breaking, 
with (iff) = vi, (H 2 ) = i>2, the remaining states have the following mass terms, 



W = d a m d ap d c p + e c a m e a/3 e p + n c a m° p vp 



+<m u tj u j + -n^M^n) 



Here m d , m e and mP are 4x4 matrices, 
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whereas m u and m N are diagonal 3x3 matrices, 



m u = 



( 



hi±v 2 





o \ 

h u 22 v 2 
h^v 2 ) 



m 



( 



V 



UN V N 

a n m, 






UN v n 
U 22 M* 





h 






Nvjf. 
33 M, ) 



(7) 



In the matrices m d , m e and m D we have neglected corrections 0(vn/M*). The diagonal 
elements satisfy four GUT relations which correspond to the unbroken SU(5), flipped 
SU(5) and Pati-Salam subgroups of SO(10). 

The crucial feature of the matrices m d , m e and m D are the mixings between the six 
brane states and the two bulk states. The first three rows of the matrices are proportional 
to the electroweak scale. The corresponding Yukawa couplings have to be hierarchical 
in order to obtain a realistic spectrum of quark and lepton masses. This corresponds to 
different strengths of the Yukawa couplings at the different fixpoints of the orbifold. The 
fourth row, proportional to M d , M l and vn, is of order the unification scale and, we 
assume, non-hierarchical. 

The mass matrices m d , m e and mP are of the form 
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(8) 



where n^Hi = 0{v 1}2 ) and Mj = 0(A GUT ). To diagonalize the matrix m it is convenient 
to define a set of four- dimensional unit vectors as follows, 



(Mi, . . . M 4 ) = Me 4 , e a ep = e ai e^ = 5 a/3 . 
Using the orthogonal matrices (a, f3 — 1 . . . 4, % — 1 . . . 3), 

1 „ ( v 2S 

V a/ 3 = {ep) a , U a p = 5 a(3 - -~5 a4 (e4i/ij + eufJ>i)S^ + O I 



(9) 



(10) 
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we can now perform a change of basis which yields for the mass matrix, 



m > = U T mV = 
where the 3x3 matrix m is given by 




(11) 
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(12) 



Here the three- vectors e a , a — 1 . . . 4, are determined by the four- vectors e*, % = 1 ... 3, 
with (e Q )t = (ej) a . Note that m is composed of three row vectors of hierarchical length, 
a structure familiar from lopsided fermion mass models. 

The hierarchy of the row vectors suggests to perform a further change of basis such 
that all remaining mixings are small. Three orthogonal three-vectors e,, ejej = e^e^ = 
5ij, can be defined by writing the matrix m in the following form 



m = 
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(13) 
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The parameters ^ are (9(//j,/*j) and therefore again hierarchical. With respect to this 
new basis the matrix m has triangular form, 



( ^7 \i x fa/3 ^ 
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(14) 



For our discussion of mass eigenvalues and mixing angles we shall need the two matrices 
mm T and m T m, which in the basis ej are both hierarchical, 
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(16) 



Consider now the up-quark mass matrix. We concentrate on the case of large tan (5 = 
v 2 /vi ~ 50, such that h d 3 ~ h 33 . The diagonal elements of the mass matrices (4), (5), (6) 
and (7) are partially connected by the GUT relations which hold on the different branes. 
For simplicity we therefore assume universally 

A*i : A*2 : A<3 ~ m u : m c : m t . (17) 

It is well known that the hierarchy of down-quark and charged lepton masses is substan- 
tially smaller than the up-quark mass hierarchy. Given the scaling (17) of the diagonal 
elements and the structure of m d and m e this implies that the down-quark and charged 
lepton mass matrices must be dominated by the off-diagonal elements. Hence, we assume 
again universally, 

A*i < V-\ ~ A*l , A*2 < 7*2 ~ A*2 , A*3 ~ 7*3 • ( 18 ) 

The parameters ^ 2 of the matrix m are then dominated by the mixing terms 7*1,2, i-e. 

7*1,2 ~ ^1,2- 

Since the up-quark matrix m" is diagonal the CKM quark mixing matrix is given by 
the matrix V which diagonalizes m d m dT . From eq. (15) one reads off for the two larger 

masses 

m b ~ 7I3 , m s ~ 7*2 , (19) 

and for the mixing angles 

K s = e c ~^, v^~^, v^~^. (20) 

A*2 A*3 A*3 

Using mt, m s and 6 C ~ 0.2 as input one obtains for the two remaining mixing angles 
V cb ~ — ~ 2 x 10- 2 , K fc ~ 9 C — ~ 4 x 10" 3 , (21) 

in agreement with analyses of weak decays [24] up to a factor of two, which is beyond 
the predictivity of our approach. 

The smallest eigenvalue vanishes in the limit //i,/i2 - ► 0, since in this case two 
vectors of the matrix m become parallel, with (5 — a and 7 = 0. Choosing, for simplicity, 
A*i/7*i < ^2/^2, one has for non-zero /ii,/i 2 , 

^2 m c m b 

7 ~ — ~ ~ 0.1 . {22) 

H2 m t m s 

This relation will also be important in our analysis of the neutrino masses. For the 
down-quark mass one obtains 



consistent with data [1]. 

The charged lepton mass matrix rrf is very similar to the down-quark mass matrix. 
The main difference is that now there are large mixings between the 'left-handed' states 
ej. To obtain the contribution of the charged leptons to the leptonic mixing matrix 
we consider the matrix m eT m e as given in eq. (16) in the basis e^. For the two large 
eigenvalues of rrf one has m T ~ p 3 ~ m& and m M ~ p 2 ~ m s . These relations are 
consistent with data within our accuracy. A potential problem is the smallness of the 
electron mass, i.e. m e /m M ~ 0.1 rrid/m s . The smallest eigenvalue of rrf is again given by 
m e /m^ ~ (fi2p>i / fil) ■ However, in our model the usual SU(5) relations don't hold for the 
second row of the mass matrices. Hence, the electron mass is not determined by down 
quark masses. 

Using the diagonal and off-diagonal elements of the mass matrices as determined from 
up- and down-quark mass matrices, we can now discuss the implications for neutrino 
masses. The heavy Majorana neutrinos scale like up-quarks (cf. (7)), 
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The light neutrino masses are given by the seesaw relation 
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(25) 



The structure of the charged lepton and the Dirac neutrino mass matrices (cf. (5), (6)) 
is the same. Both matrices lead to large mixings between the 'left-handed' states. In 
order to determine the leptonic mixing matrix we discuss the Dirac neutrino matrix in 
the basis ei where the remaining mixings of the left-handed charged leptons is small by 
construction (cf. (16)). 

The Dirac neutrino mass matrix can be written as (cf. (12)), 
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(26) 



Here the parameters pi, pi are expected to have the same hierarchy as pi, pi. However, in 
general these parameters will differ by factors 0(1) since there the entries of rrf and m D 
arise from different Yukawa couplings in the superpotential. This implies for the matrix 
rh D , with respect to the vectors e^, 

I p^AeJ + DeT^^ \ 



m D = 



p 2 (BeJ + EeT + e^ 
\ p 3 (Cef + Fe 2 r + eJ) ) 
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(27) 



where p { ~ pi. Hence, with respect to the basis p i the matrix m D has no longer triangular 
form, 



1 A Pi B Pi Pi ^ 

Cp2 D P2 P2 
V E P3 F P~3 P~3 J 



(28) 



Generically, the parameters A. . . F are all 0(1). All we know is that for /i ii2 = p 1)2 = 
the first two row vectors are parallel, with A = B = C = and D = E. For /i li2 , p li2 ^ 
one has analogous to the charged lepton mass matrix (cf. (22)), 



A, B, C, D - E ~ ^ ~ ^ ~ 7 ~ 0.1 . 

P2 P2 

From eqs. (25) and (28) one now obtains for the light neutrino mass matrix, 
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Using eq. (29) one immediately sees the order of magnitude of the different entries, 
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where m3 is the largest neutrino mass, i.e. vrt\ < m 2 < m3. It is well known that such a 
matrix can account for all neutrino data. It has previously been derived based on a U(l) 
family symmetry [16, 17] and also by requiring a compensation between the Dirac and 
Majorana neutrino mass hierarchies [25, 26]. 

Consider now the parameters in the matrix (30). The mass matrices m d , m e and 
m D have the same structure with large off-diagonal entries. For simplicity, we therefore 
assume for the mass parameters Ji i and ~p i have a similar hierarchy, approximately given 
by the down-quark masses, i.e. pi : p 2 '■ P3 ~ m d '■ m s '■ m b- One then obtains 

,2 S/l -_ ^ Ms 



0.2, 



m\m u 



0.2 . 



(32) 



p'2 M 3 m 2 s m t 

M 2 pi m\m c Mi p% 

This corresponds to the picture of sequential heavy neutrino dominance [27]. It yields 
large 2-3 mixing, sin20 23 ~ 1. The largest neutrino mass is m 3 ~ m^/M 3 . Identifying m 3 
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with \J Aml tm ~ 0.05 eV one obtains for the heavy Majorana masses M 3 ~ 10 15 GeV, 
M 2 ~ 3 x 10 12 GeV and M 1 ~ 10 10 GeV. The second neutrino mass is m 2 ~ 0.01 eV, 
which is consistent with data within our accuracy. 

Since the 2-3 determinant is small the matrix (30) can also account for the LMA 
MSW-solution of the solar neutrino problem [20]. As all neutrino masses are rather close 
to each other, with unknown coefficients 0(1), a precise prediction of the mixing angle 
O12 and the smallest neutrino mass is not possible. Generically, one has sin26i2 ~ 
7m 3 /m 2 and mi = O^ms, m 2 ). On the other hand, a definitive prediction of the matrix 
(30) is a rather large 1-3 mixing angle, 6i 3 ~ 7 ~ 0.1. 

Decays of the lightest right-handed neutrinos may be the origin of the baryon asym- 
metry of the universe [28]. In addition to the mass M 1 ~ 10 10 GeV the relevant quantities 
are the CP-asymmetry e 1 and the effective neutrino mass m = (m D ^m D ) n /M 1 . One eas- 
ily obtains e\ ~ 0.1 M 1 /M 3 ~ 10~ 6 and mi ~ 0.2 m 3 . These are the typical parameters 
of thermal leptogenesis [29] . 

Starting from three sequential families located at three different fixpoints of an orb- 
ifold, we have shown that the mixing with split bulk multiplets can lead to a characteristic 
pattern of quark and lepton mass matrices which can account for small quark mixings 
together with large lepton mixings in the charged current. Correspondingly, the quark 
mass hierarchies are large whereas the small neutrino mass hierarchy follows from the 
difference of down-quark and up-quark mass hierarchies. The dynamical origin of the 
hierarchy of Yukawa couplings at the different branes remains to be understood. 

We would like to thank A. Hebecker and D. Wyler for helpful discussions. 
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